Apparatus for analyzing music using sounds of instruments

ABSTRACT

An apparatus for analyzing music based on sound information of instruments is provided. The apparatus uses sound information of instruments or the sound information and score information in order to analyze digital sounds. The sound information of instruments performed to generate digital sounds is previously stored by pitches and strengths so that monophonic notes and polyphonic notes performed by the instruments can be easily analyzed. In addition, by using the sound information, of instruments and score information together, input digital sounds can be accurately analyzed and can be detected in the form of quantitative data.

TECHNICAL FIELD

The present invention relates to an apparatus for analyzing music basedon sound information of instruments, and more particularly, to anapparatus for analyzing music input in the form of digital sound bycomparing frequency components of input digital sound signals withfrequency components of sound information of instruments previouslystored by pitches and strengths.

BACKGROUND ART

Since personal computers started to be spread in the 1980's, computertechnology, performance and environments have been rapidly developed. Inthe 1990's, Internet was rapidly spread to various departments ofcompanies and personal life. Therefore, use of computers has been veryimportant in every field throughout the world in the 21st century, andtechniques of applying computers to the field of music have been alsodeveloped. In particular, technology of music analysis using computertechnology and digital signal processing technology has been developedin various viewpoints, but satisfactory results have never beenobtained.

DISCLOSURE OF THE INVENTION

The present invention provides an apparatus for analyzing music input inthe form of digital sounds, by which sound information of instrumentsare previously stored by pitches and strengths and frequency componentsof input digital sound signals are compared with frequency components ofthe previously stored sound information of instruments so that the moreaccurate result of analyzing the music performance can be obtained andthe analyzed result can be extracted in the form of quantitative data.

The present invention also provides an apparatus for analyzing musicinput in the form of digital sounds based on sound information ofinstruments previously stored by pitches and strengths and scoreinformation on a score to be performed.

According to an aspect of the present invention, there is provided anapparatus for analyzing music. The apparatus includes a soundinformation storage unit, which separately stores sound information bytypes of instruments; a sound information selection unit, which selectssound information of a particular instrument from the sound informationof different types of instruments stored in the sound informationstorage unit and outputs the selected sound information; a digital soundinput unit, which receives externally performed music and converts itinto a digital sound signal; a frequency analysis unit, which receivesthe digital sound signal from the digital sound input unit, decomposesit into frequency components, and outputs the frequency components inunits of frames; a comparison/analysis unit, which receives the soundinformation output from the sound information selection unit and thefrequency components output from the frequency analysis unit in units offrames, selects a lowest peak frequency from peak frequencies of thefrequency components in each frame output from the frequency analysisunit, and detects sound information including the lowest peak frequencyfrom the sound information output from the sound information selectionunit; a monophonic component detection unit, which receives the detectedsound information, the frequency components of the digital sound signal,and the lowest peak frequency from the comparison/analysis unit anddetects, as a monophonic component, sound information that has peakinformation most similar to the lowest peak frequency in the soundinformation; a monophonic component removing unit, which receives thelowest peak frequency that has been used to detect the monophoniccomponent and the frequency components of the digital sound signal fromthe monophonic component detection unit, removes the lowest peakfrequency from the frequency components, and transmits the result of theremoval to the comparison/analysis unit; a performance sound informationdetection unit, which combines monophonic components, which have beendetected by the monophonic component detection unit, to detectperformance sound information; and a performance sound informationoutput unit, which outputs the performance sound information.

According to another aspect of the present invention, there is providedan apparatus for analyzing music. The apparatus includes a soundinformation storage unit, which separately stores sound information bytypes of instruments; a sound information selection unit, which selectssound information of a particular instrument from the sound informationof different types of instruments stored in the sound informationstorage unit and outputs the selected sound information; a scoreinformation storage unit, which stores information on a score to beperformed by a particular instrument, i.e., score information; a digitalsound input unit, which receives externally performed music and convertsit into a digital sound signal; a frequency analysis unit, whichreceives the digital sound signal from the digital sound input unit,decomposes it into frequency components, and outputs the frequencycomponents in units of frames; an expected performance value generationunit, which commences an operation in response to an external controlsignal, generates expected performance values in units of frames basedon the score information stored in the score information storage unit astime lapses since it commenced the operation, and outputs the expectedperformance value in units of frames; a comparison/analysis unit, whichreceives the sound information output from the sound informationselection unit, the frequency components output in units of frames fromthe frequency analysis unit, and the expected performance values outputfrom the expected performance value generation unit, selects a lowestexpected performance value from expected performance values that havenot been compared with the frequency components, detects soundinformation corresponding to the lowest expected performance value, anddetermines whether the detected sound information corresponding to thelowest expected performance value is included in the frequencycomponents; a monophonic component detection unit, which receives thesound information corresponding to the lowest expected performance valueand the frequency components and when the comparison/analysis unitdetermines that the sound information corresponding to the lowestexpected performance value is included in the frequency components,detects the received sound information as a monophonic component; amonophonic component removing unit, which receives the monophoniccomponent and the frequency components of the digital sound signal fromthe monophonic component detection unit, removes the monophoniccomponent from the frequency components, and transmits the result of theremoval to the comparison/analysis unit; a performance sound informationdetection unit, which combines monophonic components, which have beendetected by the monophonic component detection unit, to detectperformance sound information; and a performance sound informationoutput unit, which outputs the performance sound information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing examples of sound information ofinstruments.

FIG. 2 is a schematic block diagram of an apparatus for analyzing musicaccording to a first embodiment of the present invention.

FIG. 3 is a flowchart of a procedure of analyzing music using anapparatus for analyzing music according to the first embodiment of thepresent invention.

FIG. 3A is a flowchart of a procedure of detecting monophonicinformation of a frame using an apparatus for analyzing music accordingto the first embodiment of the present invention.

FIG. 3B is a flowchart of a procedure of comparing and analyzingfrequency components of a frame using an apparatus for analyzing musicaccording to the first embodiment of the present invention.

FIGS. 4A through 4C are diagrams showing the waveforms of frequencies inorder to explain a procedure in which a monophonic note is detected froma plurality of performing notes using an apparatus for analyzing musicaccording to the first embodiment of the present invention.

FIG. 5 is a schematic block diagram of an apparatus for analyzing musicaccording to a second embodiment of the present invention.

FIG. 6 is a flowchart of a procedure of analyzing music using anapparatus for analyzing music according to the second embodiment of thepresent invention.

FIG. 6A is a flowchart of a procedure of detecting monophonicinformation and performance error information of a current frame usingan apparatus for analyzing music according to the second embodiment ofthe present invention.

FIGS. 6B and 6C are flowcharts of a procedure of performing comparisonand analysis on frequency components of the frame using an apparatus foranalyzing music according to the second embodiment of the presentinvention.

FIG. 6D is a flowchart of a procedure of correcting an expectedperformance value using an apparatus for analyzing music according tothe second embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, preferred embodiments of an apparatus for analyzing musicaccording to the present invention will be described in detail withreference to the attached drawings.

FIG. 1 is a diagram showing examples of sound information ofinstruments. FIG. 1 shows that sound information is different amongdifferent types of musical instruments. Sound information (a) expressesa piano sound at a pitch C5. Sound information (b) expresses a trumpetsound at a pitch C5. Sound information (c) expresses a violin sound at apitch C5. Sound information (d) expresses a female vocal sound at apitch C5.

Referring to FIG. 1( a), since a hammer hit a line when a keyboard ispressed, the strength of a piano sound increases throughout the entirefrequency region and each frequency component appears clearly. In themeantime, as time lapses, the strength of a piano sound decreasesrapidly.

Referring to FIG. 1( b), due to the characteristics of a windinstrument, a trumpet sound has thin and clear harmonic components.However, as harmonics gets higher, vibration gradually occurs little bylittle.

Referring to FIG. 1( c), due to the characteristics of a stringinstrument, a violin sound has frequency components spread up and down.As harmonics gets higher, frequency spread appears clearly.

Referring to FIG. 1( d), due to inaccuracy of a tone, a female vocalsound has frequency components vibrating largely and does not have manyharmonic components.

By applying the fact that sound information is different among differenttypes of instruments even if the same pitch is performed, as describedabove, accurate analysis results can be obtained.

FIG. 2 is a schematic block diagram of an apparatus for analyzing musicaccording to a first embodiment of the present invention. Referring toFIG. 2, the apparatus for analyzing music according to the firstembodiment includes a sound information storage unit 10, a digital soundinput unit 110, a frequency analysis unit 120, a comparison/analysisunit 130, a monophonic component detection unit 140, a monophoniccomponent removing unit 150, a performance sound information detectionunit 160, a performance sound information output unit 170, and a soundinformation selection unit 180.

The sound information storage unit 10 separately stores soundinformation by types of instruments. The sound information selectionunit 180 selects sound information “A” of a desired instrument from thesound information of different types of instruments stored in the soundinformation storage unit 10 and outputs the selected sound information“A”. Here, the sound information storage unit 10 stores the soundinformation in the form of wave data or the strengths of differentfrequency components. In a case where the sound information is stored inthe form of wave data, if the sound information selection unit 180generates a sound information request, the sound information storageunit 10 detects frequency components of a requested sound from the wavedata and provides them.

The digital sound input unit 110 receives externally performed music andconverts it into a digital sound signal. The frequency analysis unit 120receives the digital sound signal from the digital sound input unit 110,decomposes it into frequency components “F” in units of frames, andoutputs the frequency components “F” in units of frames.

The comparison/analysis unit 130 receives the sound information “A” thatis output from the sound information selection unit 180 and thefrequency components “F” that are output from the frequency analysisunit 120 in units of frames and compares them. More specifically, thecomparison/analysis unit 130 selects a lowest peak frequency “F_(PL1)”from the peak frequencies of the frequency components “F” in a singleframe output from the frequency analysis unit 120, and detects soundinformation “A_(PL1)” including the lowest peak frequency “F_(PL1)” inthe sound information “A” output from the sound information selectionunit 180.

The monophonic component detection unit 140 receives the detected soundinformation “A_(PL1)”, the frequency components “F”, and the lowest peakfrequency “F_(PL1)” from the comparison/analysis unit 130, and detects,as a monophonic component “A_(S)”, sound information that has peakinformation most similar to the lowest peak frequency “F_(PL1)” in thesound information “A_(PL1)”.

In the meantime, the monophonic component detection unit 140 detectstime information of each frame and then detects the pitch and strengthof each monophonic note included in each frame. In addition, when thedetected monophonic component “A_(S)” is a new one that has not beenincluded in the previous frame, the monophonic component detection unit140 divides the current frame including the new monophonic component“A_(S)” into a plurality of subframes, founds out a subframe includingthe new monophonic component “A_(S)”, and detects time information ofthe found subframe together with the monophonic component “A_(S)”, i.e.,pitch and strength information.

The monophonic component removing unit 150 receives the lowest peakfrequency “F_(PL1)” and the frequency components “F” from the monophoniccomponent detection unit 140, removes the lowest peak frequency“F_(PL1)” from the frequency components “F”, and transmits the result ofthe removal (F←F-F_(PL1)) to the comparison/analysis unit 130.

Then, the comparison/analysis unit 130 determines whether the frequencycomponents “F” received from the monophonic component removing unit 150include effective peak frequency information. When it is determined thateffective peak frequency information is included in the frequencycomponents “F” received from the monophonic component removing unit 150,the comparison/analysis unit 130 selects a lowest peak frequency“F_(PL2)” from the frequency components “F” and detects soundinformation “A_(PL2)” including the lowest peak frequency “F_(PL2)”.However, when it is determined that effective peak frequency informationis not included in the frequency components “F” received from themonophonic component removing unit 150, the comparison/analysis unit 130receives frequency components of a next frame from the frequencyanalysis unit 120, selects a lowest peak frequency from peak frequenciesincluded in the received frequency components, and detects soundinformation including the lowest peak frequency, as described above. Inother words, until all monophonic information included in a currentframe is detected, the frequency components “F” of the current frameoutput from the frequency analysis unit 120 are compared with soundinformation transmitted from the sound information selection unit 180 tobe analyzed while sequentially and repeatedly processed by thecomparison/analysis unit 130, the monophonic component detection unit140, and the monophonic component removing unit 150.

The performance sound information detection unit 160 combines monophoniccomponents “A_(S)”, which have been detected by the monophonic componentdetection unit 140, to detect performance sound information. It isapparent that the performance sound information detection unit 160 candetect performance sound information even if polyphonic notes areperformed. The performance sound information detection unit 160 detectsinformation on individual monophonic notes included in performance soundof polyphonic notes and combines the detected monophonic information soas to detect performance sound information corresponding to thepolyphonic notes.

The performance sound information output unit 170 outputs theperformance sound information detected by the performance soundinformation detection unit 160.

FIGS. 3 through 3B are flowcharts of a method performed by an apparatusfor analyzing music according to the first embodiment of the presentinvention.

FIG. 3 is a flowchart of a procedure of analyzing music using anapparatus for analyzing music according to the first embodiment of thepresent invention. Referring to FIG. 3, after sound information ofdifferent types of instruments is generated and stored (not shown),sound information of a particular instrument to be actually played isselected from the stored sound information of different types ofinstruments in step s100.

Next, if a digital sound signal is input in step s200, the digital soundsignal is decomposed into frequency components in units of frames instep s400. The frequency components of the digital sound signal arecompared with the frequency components of the selected sound informationof the particular instrument and analyzed to detect monophonicinformation from the digital sound signal in units of frames in steps500. The detected monophonic information is output in step s600.

Steps s200 through s600 are repeated until the input of the digitalsound signal is stopped or an end command is input in step s300.

FIG. 3A is a flowchart of step s500 of detecting the monophonicinformation of each frame using an apparatus for analyzing musicaccording to the first embodiment of the present invention. Referring toFIG. 3A, time information of a current frame is detected in step s510.The frequency components of the current frame are compared with thefrequency components of the selected sound information of the particularinstrument and analyzed so as to detect the pitch, strength, and timeinformation of each of monophonic notes included in the current frame instep s520. The detected pitch, strength, and time information compose adetected monophonic component in step s530.

If it is determined that the monophonic note detected in step s530 is anew one which is not included in the previous frame in step s540, thecurrent frame is divided into a plurality of subframes in step s550. Asubframe including the new monophonic note is detected from theplurality of subframes in step s560. Time information of the detectedsubframe is detected in step s570. The time information of the subframeis set as the time information of the current monophonic note in steps580. Steps s540 through s580 can be omitted when the detectedmonophonic note is in a low frequency range, i.e. minimum number ofsamples to detect the note frequency is greater than subframe size, orwhen the accuracy of time information is not required.

FIG. 3B is a flowchart of step s520 of comparing and analyzing thefrequency components of the current frame using an apparatus foranalyzing music according to the first embodiment of the presentinvention. Referring to FIG. 3B, a lowest peak frequency included in thecurrent frame of the input digital sound signal is selected in steps521. Next, sound information including the selected peak frequency isdetected from the sound information of the particular instrument in steps522. In the sound information detected in step s522, sound informationhaving most similar peak information to the component of the selectedpeak frequency is detected as monophonic information in step s523.

After the monophonic information corresponding to the lowest peakfrequency is detected, the frequency components included in the detectedmonophonic information are removed from the frequency componentsincluded in the current frame in step s524. Thereafter, it is determinedwhether there is any peak frequency component in the current frame instep s525. If it is determined that there is any peak frequencycomponent in the current frame, steps s521 through s524 are repeated.

FIGS. 4A through 4C are diagrams showing the waveforms of frequencies inorder to explain a procedure in which a monophonic note is detected froma plurality of performing notes using an apparatus for analyzing musicaccording to the first embodiment of the present invention. The X axisindicates a pitch, i.e., a fast Fourier transform (FFT) index, and the Yaxis indicates the strength of each frequency component, i.e., amagnitude as the result of FFT.

Step s520 will be described in more detail with reference to FIGS. 4Athrough 4C.

In FIG. 4A, a waveform (a) shows a case where the current frame of theinput digital sound signal includes three notes D3, F3#, and A3. In thiscase, the fundamental frequency component of the note D3 is selected asthe lowest peak frequency component among the peak frequency componentsincluded in the current frame in step s521. In the sound information ofthe particular instrument, sound information including the fundamentalfrequency component of the note D3 is detected in step s522. In steps522, sound information of many notes, such as D3, D2, and A1, can bedetected.

Then, in the sound information detected in step s522, the soundinformation of the note D3 having a most similar peak frequencycomponent to the peak frequency component selected in step s521 isdetected as monophonic information of the selected peak frequencycomponent in step s523. The monophonic information of the note D3 isshown in a waveform (b) in FIG. 4A.

Thereafter, the monophonic information of the note D3 (FIG. 4A(b)), isremoved from the frequency components of the notes D3, F3#, and A3included in the current frame of the digital sound signal in step s524.

Then, the frequency components of the notes F3# and A3, as shown in FIG.4A(c), remain in the current frame. Steps s521 through s524 are repeateduntil there remains no frequency component in the current frame so thatmonophonic information of all notes included in the current frame can bedetected.

In the above case, monophonic information of all notes D3, F3#, and A3can be detected by repeating steps s521 through s524 three times.

FIG. 4B is a diagram for explaining a procedure of detecting andremoving the note F3# in the above case. FIG. 4B(a) shows the frequencycomponents of the notes F3# and A3 remaining in the sound information ofthe current frame after removing the note D3 from the notes D3, F3#, andA3. FIG. 4B(b) shows the frequency components of the note F3# detectedthrough the above steps. FIG. 4B(c) shows the frequency components ofthe note A3 remaining after removing the note F3# (FIG. 4B(b)) from thewaveform shown in FIG. 4B(a).

FIG. 4C is a diagram for explaining a procedure of detecting andremoving the note A3 in the above case. FIG. 4C(a) shows the frequencycomponents of the notes A3 remaining in the sound information of thecurrent frame after removing the note F3# from the notes F3# and A3.FIG. 4C(b) shows the frequency components of the note A3 detectedthrough the above steps. FIG. 4C(c) shows remaining frequency componentsafter removing the note A3 (FIG. 4C(b)) from the waveform shown in FIG.4C(a). Since all of the three performing notes have been detected, theremaining frequency components have strength near zero. Accordingly, theremaining frequency components are considered as being caused by nose.

FIG. 5 is a schematic block diagram of an apparatus for analyzing musicaccording to a second embodiment of the present invention.

In the second embodiment of the present invention, sound information ofan instrument and information of a score to be performed are used. Ifall information of each note having different frequency components canbe constructed into the sound information of each instrument, an inputdigital sound signal can be accurately analyzed. However, actually,since it is difficult to construct all information of each note into thesound information of each instrument, the second embodiment of thepresent invention is provided to overcome this problem. In other words,in the second embodiment of the present invention, score information ofa musical performance is detected, notes to be input are predicted basedon the sound information of a particular instrument and the scoreinformation, and input digital sound is analyzed using information onthe predicted notes.

Referring to FIG. 5, the apparatus for analyzing music according to thesecond embodiment of the present invention includes a sound informationstorage unit 10, a score information storage unit 20, a digital soundinput unit 210, a frequency analysis unit 220, a comparison/analysisunit 230, a monophonic component detection unit 240, a monophoniccomponent removing unit 250, an expected performance value generationunit 290, a performance sound information detection unit 260, aperformance sound information output unit 270, and a sound informationselection unit 280.

The sound information storage unit 10 separately stores soundinformation by types of instruments. The sound information selectionunit 280 selects sound information “A” of a desired instrument from thesound information of different types of instruments stored in the soundinformation storage unit 10 and outputs the selected sound information“A”. Here, the sound information storage unit 10 stores the soundinformation in the form of wave data or as the strengths of differentfrequency components. In a case where the sound information is stored inthe form of wave data, if the sound information selection unit 280generates a sound information request, the sound information storageunit 10 detects frequency components of a requested sound from the wavedata and provides them.

The score information storage unit 20 stores information on a score tobe performed by a particular instrument. The score information storageunit 20 stores and manages at least one type of information among pitchinformation, note length information, tempo information, rhythmicinformation, note strength information, detailed performance information(e.g., staccato, staccatissimo, and pralltriller), and discriminationinformation for performance using both hands or performance using aplurality of instruments, based on the score to be performed.

The digital sound input unit 210 receives externally performed music andconverts it into a digital sound signal. The frequency analysis unit 220receives the digital sound signal from the digital sound input unit 210,decomposes it into frequency components “F” in units of frames, andoutputs the frequency components “F” in units of frames.

The expected performance value generation unit 290 commences anoperation when music sound is input through the digital sound input unit210, generates expected performance values “E” in units of frames basedon the score information stored in the score information storage unit 20as time lapses since it commenced the operation, and outputs theexpected performance value “E” in units of frames.

The comparison/analysis unit 230 receives the sound information “A”output from the sound information selection unit 280, the frequencycomponents “F” output from the frequency analysis unit 220 in units offrames, and the expected performance values “E” output from the expectedperformance value generation unit 290; selects a lowest expectedperformance value “E_(L1)” from the expected performance values “E” thathave not been compared with the frequency components “F”; detects soundinformation “A_(L1)” corresponding to the lowest expected performancevalue “E_(L1)”; and determines whether the sound information “A_(L1)” isincluded in the frequency components “F”.

The monophonic component detection unit 240 receives the soundinformation “A_(L1)” corresponding to the lowest expected performancevalue “E_(L1)” and the frequency components “F”. When thecomparison/analysis unit 230 determines that the sound information“A_(L1)” is included in the frequency components “F”, the monophoniccomponent detection unit 240 detects the sound information “A_(L1)” as amonophonic component “A_(S)”.

In the meantime, the monophonic component detection unit 240 detectstime information of each frame and the pitch and strength of eachmonophonic note included in each frame. In addition, when the detectedmonophonic component “A_(S)” is a new one that has not been included inthe previous frame, the monophonic component detection unit 240 dividesthe current frame including the new monophonic component “A_(S)” into aplurality of subframes, founds out a subframe including the newmonophonic component “A_(S)”, and detects time information of the foundsubframe together with the monophonic component “A_(S)”, i.e., pitch andstrength information.

When the comparison/analysis unit 230 determines that the soundinformation “A_(L1)” is not included in the frequency components “F”,the monophonic component detection unit 240 detects historicalinformation indicating how many consecutive frames the sound information“A_(L1)” is included in and when the sound information “A_(L1)” is notincluded in a predetermined number of consecutive frames, removes thesound information “A_(L1)” from the expected performance values “E”.

The monophonic component removing unit 250 receives the monophoniccomponent “A_(S)” and the frequency components “F” from the monophoniccomponent detection unit 240, removes the monophonic component “A_(S)”from the frequency components “F”, and transmits the result of theremoval (F←F-A_(S)) to the comparison/analysis unit 230.

In the meantime, when expected performance values with respect to aframe for which frequency components are generated by the frequencyanalysis unit 220 are not generated by the expected performance valuegeneration unit 290, the comparison/analysis unit 230 receives the soundinformation “A” output from the sound information selection unit 280 andthe frequency components “F” output from the frequency analysis unit 220in units of frames. Then, the comparison/analysis unit 230 selects alowest peak frequency “F_(PL)” from the peak frequencies of thefrequency components “F” in a current frame and detects soundinformation “A_(PL)” including the lowest peak frequency “F_(PL)” in thesound information “A” output from the sound information selection unit280.

The monophonic component detection unit 240 receives the soundinformation “A_(PL)”, the frequency components “F”, and the lowest peakfrequency “F_(PL)” from the comparison/analysis unit 230, and detects,as performance error information “Er”, sound information “A_(F)” thathas peak information most similar to the lowest peak frequency “F_(PL)”in the sound information “A_(PL)”. In addition, the monophonic componentdetection unit 240 searches the score information and determines whetherthe performance error information “Er” is included in notes to beperformed next in the score information. If it is determined that theperformance error information “Er” is included in the notes to beperformed next in the score information, the monophonic componentdetection unit 240 adds the performance error information “Er” to theexpected performance values “E” and outputs sound informationcorresponding to the performance error information “Er” as a monophoniccomponent “A_(S)”. If it is determined that the performance errorinformation “Er” is not included in the notes to be performed next inthe score information, the monophonic component detection unit 240outputs the sound information corresponding to the performance errorinformation “Er” as an error note component “E_(S)”.

When the error note component “E_(S)” is detected by the monophoniccomponent detection unit 240, the monophonic component removing unit 250receives the error note component “E_(S)” and the frequency components“F” from the monophonic component detection unit 240, removes the errornote component “E_(S)” from the frequency components “F”, and transmitsthe result of the removal (F←F-E_(S)) to the comparison/analysis unit230.

Then, the comparison/analysis unit 230 determines whether the frequencycomponents “F” received from the monophonic component removing unit 250include effective peak frequency information. When it is determined thateffective peak frequency information is included in the frequencycomponents “F” received from the monophonic component removing unit 250,the comparison/analysis unit 230 performs the above described operationon the frequency components “F” received from the monophonic componentremoving unit 250. However, when it is determined that effective peakfrequency information is not included in the frequency components “F”received from the monophonic component removing unit 250, thecomparison/analysis unit 230 receives frequency components of a nextframe of the input digital sound signal from the frequency analysis unit220 and performs the above described operation on the frequencycomponents of the next frame.

The performance sound information detection unit 260 and the performancesound information output unit 270 perform the same functions as theperformance sound information detection unit 160 and the performancesound information output unit 170 in the first embodiment of the presentinvention, and thus detailed descriptions thereof will be omitted.

FIG. 6 is a flowchart of a procedure of analyzing music using anapparatus for analyzing music according to the second embodiment of thepresent invention.

The following description concerns a procedure of analyzing externallyinput digital sound based on sound information of different types ofinstruments and score information using an apparatus for analyzing musicaccording to the second embodiment of the present invention.

After sound information of different types of instruments and scoreinformation of music to be performed are generated and stored (notshown), sound information of a particular instrument to be actuallyplayed and score information of music to be actually performed areselected from the stored sound information of different types ofinstruments and score information in steps t100 and t200. A method ofgenerating the score information of music to be performed is beyond thescope of the present invention. At present, there are many techniques ofscanning a score printed on paper, converting the scanned score intoperformance information of music instrument digital interface (MIDI)music, and storing the performance information. Thus, a detaileddescription of generating and storing the score information will beomitted.

The score information includes, for example, pitch information, notelength information, tempo information, rhythmic information, notestrength information, detailed performance information (e.g., staccato,staccatissimo, and pralltriller), and discrimination information forperformance using both hands or performance using a plurality ofinstruments.

After the sound information and the score information are selected insteps t100 and t200, if a digital sound signal is input in step t300,the digital sound signal is decomposed into frequency components inunits of frames in step t500. The frequency components of the digitalsound signal are compared with the selected score information and thefrequency components of the selected sound information and analyzed soas to detect performance error information and monophonic information ofa current frame from the digital sound signal in step t600.

Thereafter, the detected monophonic information is output in step t700.

Performance accuracy can be estimated based on the performance errorinformation in step t800. If the performance error informationcorresponds to a note (for example, a variation) intentionally performedby a player, the performance error information is added to the scoreinformation in step t900. The steps t800 and t900 can be selectivelyperformed.

FIG. 6A is a flowchart of step t600 of detecting the monophonicinformation and the performance error information of the current frameusing an apparatus for analyzing music according to the secondembodiment of the present invention. Referring to FIG. 6A, timeinformation of the current frame is detected in step t610. The frequencycomponents of the current frame are compared with the frequencycomponents of the selected sound information of the particularinstrument and with the score information and are analyzed to detectpitch, strength and time information of each monophonic note included inthe current frame in step t620. In step t640, as a result of theanalysis, monophonic information and performance error information aredetected with respect to the current frame.

If it is determined that a monophonic note corresponding to the detectedmonophonic information is a new one that is not included in the previousframe in step t650, the current frame is divided into a plurality ofsubframes in step t660. Among the plurality of subframes, a subframewhich includes the new monophonic note is detected in step t670. Timeinformation of the detected subframe is detected in step t680. The timeinformation of the subframe is set as the time information of themonophonic information in step t690. Similar to the first embodiment,the steps t650 through t690 can be omitted either when the monophonicnote is in a low frequency range or when the accuracy of timeinformation is not required.

FIGS. 6B and 6C are flowcharts of step t620 of performing comparison andanalysis on the frequency components of the current frame using anapparatus for analyzing music according to the second embodiment of thepresent invention. Referring to FIGS. 6B and 6C, in step t621, withrespect to the digital sound signal, which is generated in real time asthe particular instrument is performed, expected performance values ofthe current frame are generated, and it is determined whether there isany expected performance value which has not been compared with realperformance sound, i.e., the digital sound signal, in the current frame.

If it is determined that there is no expected performance value whichhas not been compared with the digital sound signal in the current framein step t621, it is determined whether the frequency components of thedigital sound signal in the current frame correspond to performanceerror information; performance error information and monophonicinformation are detected; and the frequency components of soundinformation, which corresponds to the performance error information andthe monophonic information, are removed from the digital sound signal inthe current frame, in steps t622 through t628.

More specifically, a lowest peak frequency of the input digital soundsignal in the current frame is selected in step t622. Sound informationcontaining the selected peak frequency is detected from the soundinformation of the particular instrument in step t623. Sound informationcontaining most similar peak information to the component of theselected peak frequency is detected from the sound information detectedin step t623, as performance error information in step t624. If it isdetermined that the performance error information is included in notes,which are expected to be performed next based on the score information,in step t625, notes corresponding to the performance error informationare added to the expected performance values in step t626. Next, theperformance error information is set as monophonic information in stept627. The frequency components of the sound information detected as theperformance error information or the monophonic information in step t624or t627 are removed from the current frame of the digital sound signalin step t628.

If it is determined that there is any expected performance value whichhas not been compared with the digital sound signal in the current framein step t621, the digital sound signal is compared with the one or moreexpected performance values and analyzed to detect monophonicinformation of the current frame, and the frequency components of soundinformation corresponding to the monophonic information are removed fromthe current frame of the digital sound signal, in steps t630 throught634.

More specifically, sound information of a lowest pitch which has notbeen compared with frequency components included in the current frame ofthe digital sound signal is selected from sound informationcorresponding to the one or more expected performance values, which havenot been compared, in step t630. If it is determined that the frequencycomponents of the sound information selected in step t630 are includedin the frequency components included in the current frame of the digitalsound signal in step t631, the selected sound information is set asmonophonic information in step t632. Then, the frequency components ofthe selected sound information are removed from the current frame of thedigital sound signal in step t633. If it is determined that thefrequency components of the selected sound information are not includedin the frequency components included in the current frame of the digitalsound signal in step t631, the one or more expected performance valuesare corrected in step t635. The steps t630 through t633 are repeateduntil it is determined that every note corresponding to the one or moreexpected performance values has compared with the digital sound signalof the current frame in step t634.

The steps t621 through t628 and t630 through t635 shown in FIGS. 6B and6C are repeated until it is determined that no peak frequency componentis left in the digital sound signal in the current frame in step t629.

FIG. 6D is a flowchart of step 635 of correcting the one or moreexpected performance values using an apparatus for analyzing musicaccording to the second embodiment of the present invention. Referringto FIG. 6D, if it is determined that the frequency components of thesound information selected in step t630 are not included in at least apredetermined number N of consecutive previous frames N in step t636,and if it is determined that the frequency components of the selectedsound information have been included in at least one previous frame ofthe digital sound signal in step t637, an expected performance valuecorresponding to the selected sound information is removed in step t639.Alternatively, if it is determined that the frequency components of theselected sound information are not included in at least thepredetermined number N of consecutive previous frames N in step t636,and if it is determined that the frequency components of the selectedsound information have never been included in any previous frame of thedigital sound signal in step t637, the selected sound information is setas the performance error information in step t638, and an expectedperformance value corresponding to the selected sound information isremoved in step t639.

The above description just concerns embodiments of the presentinvention. The present invention is not restricted to the aboveembodiments, and various modifications can be made thereto within thescope defined by the attached claims. For example, the shape andstructure of each member specified in the embodiments can be changed.

INDUSTRIAL APPLICABILITY

An apparatus for analyzing music according to the present invention usessound information or sound information and score information, therebyquickly analyzing input digital sounds and increasing the accuracy ofanalysis. In conventional approaches of analyzing digital sounds, musiccomposed of polyphonic pitches, for example, piano music, cannot beanalyzed. However, according to the present invention, as well asmonophonic pitches, polyphonic pitches contained in digital sounds canbe quickly and accurately analyzed.

Therefore, the result of analyzing digital sounds according to thepresent invention can be directly applied to an electronic score, andperformance information can be quantitatively detected using the resultof the analysis. This result of the analysis can be widely used in frommusical education for children to professional players' practice. Thatis, by using a technique of the present invention allowing input digitalsounds to be analyzed in real time, positions of currently performednotes on the electronic score are recognized in real time and positionsof notes to be performed next are automatically indicated on theelectronic score, so that players can concentrate on performance withoutcaring about turning over the leaves of a paper score.

In addition, the present invention compares performance informationobtained as the result of the analysis with previously stored scoreinformation to detect performance accuracy so that players can beinformed about wrong performance. The detected performance accuracy canbe used as data by which a player's performance is evaluated.

1. An apparatus for analyzing music, comprising: a sound informationstorage unit, which separately stores sound information by types ofinstruments; a sound information selection unit, which selects soundinformation of a particular instrument from the sound information ofdifferent types of instruments stored in the sound information storageunit and outputs the selected sound information; a digital sound inputunit, which receives externally performed music and converts it into adigital sound signal; a frequency analysis unit, which receives thedigital sound signal from the digital sound input unit, decomposes itinto frequency components, and outputs the frequency components in unitsof frames; a comparison/analysis unit, which receives the soundinformation output from the sound information selection unit and thefrequency components output from the frequency analysis unit in units offrames, selects a lowest peak frequency from peak frequencies of thefrequency components in each frame output from the frequency analysisunit, and detects sound information including the lowest peak frequencyfrom the sound information output from the sound information selectionunit; a monophonic component detection unit, which receives the detectedsound information, the frequency components of the digital sound signal,and the lowest peak frequency from the comparison/analysis unit anddetects, as a monophonic component, sound information that has peakinformation most similar to the lowest peak frequency in the soundinformation; a monophonic component removing unit, which receives thelowest peak frequency that has been used to detect the monophoniccomponent and the frequency components of the digital sound signal fromthe monophonic component detection unit, removes the lowest peakfrequency from the frequency components, and transmits the result of theremoval to the comparison/analysis unit; a performance sound informationdetection unit, which combines monophonic components, which have beendetected by the monophonic component detection unit, to detectperformance sound information; and a performance sound informationoutput unit, which outputs the performance sound information.
 2. Theapparatus of claim 1, wherein the sound information storage unit storesthe sound information of different types of instruments in the form ofwave data, and when a sound information request is generated from anexternal device, the sound information storage unit detects frequencycomponents of sound information corresponding to the sound informationrequest from the wave data and provides them.
 3. The apparatus of claim1, wherein the sound information storage unit stores the soundinformation of different types of instruments in the form of strength ofdifferent frequency components, which can be directly expressed.
 4. Theapparatus of claim 1, wherein the monophonic component detection unitdetects time information of each frame and then detects pitch andstrength of each monophonic note included in each frame.
 5. Theapparatus of claim 4, wherein when the detected monophonic component isa new one that has not been included in a previous frame, the monophoniccomponent detection unit divides the current frame including the newmonophonic component into a plurality of subframes, founds out asubframe including the new monophonic component, and detects timeinformation of the found subframe together pitch and strengthinformation of a monophonic note corresponding to each monophoniccomponent.
 6. The apparatus of claim 1, wherein when it is determinedthat the frequency components received from the monophonic componentremoving unit include effective peak frequency information, thecomparison/analysis unit selects a lowest peak frequency from theeffective peak frequency information and detects sound informationincluding the selected lowest peak frequency, and when it is determinedthat the frequency components received from the monophonic componentremoving unit does not include effective peak frequency information, thecomparison/analysis unit receives frequency components of a next framefrom the frequency analysis unit, selects a lowest peak frequency frompeak frequencies included in the received frequency components, anddetects sound information including the selected lowest peak frequency.7. An apparatus for analyzing music, comprising: a sound informationstorage unit, which separately stores sound information by types ofinstruments; a sound information selection unit, which selects soundinformation of a particular instrument from the sound information ofdifferent types of instruments stored in the sound information storageunit and outputs the selected sound information; a score informationstorage unit, which stores information on a score to be performed by aparticular instrument, i.e., score information; a digital sound inputunit, which receives externally performed music and converts it into adigital sound signal; a frequency analysis unit, which receives thedigital sound signal from the digital sound input unit, decomposes itinto frequency components, and outputs the frequency components in unitsof frames; an expected performance value generation unit, whichcommences an operation in response to an external control signal,generates expected performance values in units of frames based on thescore information stored in the score information storage unit as timelapses since it commenced the operation, and outputs the expectedperformance value in units of frames; a comparison/analysis unit, whichreceives the sound information output from the sound informationselection unit, the frequency components output in units of frames fromthe frequency analysis unit, and the expected performance values outputfrom the expected performance value generation unit, selects a lowestexpected performance value from expected performance values that havenot been compared with the frequency components, detects soundinformation corresponding to the lowest expected performance value, anddetermines whether the detected sound information corresponding to thelowest expected performance value is included in the frequencycomponents; a monophonic component detection unit, which receives thesound information corresponding to the lowest expected performance valueand the frequency components and when the comparison/analysis unitdetermines that the sound information corresponding to the lowestexpected performance value is included in the frequency components,detects the received sound information as a monophonic component; amonophonic component removing unit, which receives the monophoniccomponent and the frequency components of the digital sound signal fromthe monophonic component detection unit, removes the monophoniccomponent from the frequency components, and transmits the result of theremoval to the comparison/analysis unit; a performance sound informationdetection unit, which combines monophonic components, which have beendetected by the monophonic component detection unit, to detectperformance sound information; and a performance sound informationoutput unit, which outputs the performance sound information.
 8. Theapparatus of claim 7, wherein the monophonic component detection unitdetects time information of each frame and then detects pitch andstrength of each monophonic note included in each frame.
 9. Theapparatus of claim 8, wherein when the detected monophonic component isa new one that has not been included in a previous frame, the monophoniccomponent detection unit divides the current frame including the newmonophonic component into a plurality of subframes, founds out asubframe including the new monophonic component, and detects timeinformation of the found subframe together pitch and strengthinformation of a monophonic note corresponding to each monophoniccomponent.
 10. The apparatus of any one of claims 7 through 9, whereinwhen the comparison/analysis unit determines that the sound informationcorresponding to the lowest expected performance value is not includedin the frequency components, the monophonic component detection unitdetects historical information indicating in how many consecutive framesthe sound information corresponding to the lowest expected performancevalue is included, and when the sound information corresponding to thelowest expected performance value is not included in a predeterminednumber of consecutive frames, removes the sound informationcorresponding to the lowest expected performance value from the expectedperformance values.
 11. The apparatus of claim 10, wherein when expectedperformance values with respect to a frame for which frequencycomponents are generated by the frequency analysis unit are notgenerated, the comparison/analysis unit receives the sound informationof the particular instrument output from the sound information selectionunit and the frequency components output from the frequency analysisunit in units of frames, selects a lowest peak frequency from the peakfrequencies of the frequency components in a current frame, and detectssound information including the lowest peak frequency from the soundinformation output from the sound information selection unit.
 12. Theapparatus of claim 11, wherein the monophonic component detection unitreceives the detected sound information, the frequency components, andthe lowest peak frequency from the comparison/analysis unit, detects, asperformance error information, sound information that has peakinformation most similar to the lowest peak frequency from the soundinformation detected by the comparison/analysis unit, and adds theperformance error information to the expected performance values andoutputs sound information corresponding to the performance errorinformation as the monophonic component when it is determined that theperformance error information is included in notes to be performed nextin the score information.
 13. The apparatus of claim 12, wherein when itis determined that the performance error information is not included inthe notes to be performed next in the score information, the monophoniccomponent detection unit outputs the sound information corresponding tothe performance error information as an error note component.
 14. Theapparatus of claim 13, wherein the monophonic component removing unitreceives the error note component and the frequency components from themonophonic component detection unit, removes the error note componentfrom the frequency components, and transmits the result of the removalto the comparison/analysis unit.
 15. The apparatus of claim 13, wherein,the comparison/analysis unit receives the frequency components from themonophonic component removing unit as an input when it is determinedthat effective peak frequency information is included in the frequencycomponents received from the monophonic component removing unit andreceives frequency components of a next frame of the input digital soundsignal from the frequency analysis unit when it is determined thateffective peak frequency information is not included in the frequencycomponents received from the monophonic component removing unit.
 16. Theapparatus of claim 7, wherein the sound information storage unit storesthe sound information of different types of instruments in the form ofwave data, and when a sound information request is generated from anexternal device, the sound information storage unit detects frequencycomponents of sound information corresponding to the sound informationrequest from the wave data and provides them.
 17. The apparatus of claim7, wherein the sound information storage unit stores the soundinformation of different types of instruments in the form of strength ofdifferent frequency components, which can be directly expressed.